In conventional database systems, transactions such as update, delete, write, etc. are written to volatile memory. Periodically, the transactions and data on the volatile memory are moved to non-volatile storage such as a disk. When the transactions or data are moved from volatile memory to non-volatile storage, the data and transactions are in a state of flux. Rather than forcing every transaction to non-volatile storage immediately after a transaction is completed, transactions are written to a log (i.e., logged) on a storage device. These transactions are logged to the storage device as the transactions are occurring so that, in the event of a system failure, the log can be replayed to redo or restore the transactions, returning the database system to a state consistent with the state of the database system at the time of the failure.
To facilitate restoration or logical recovery of a database system, the database system generates a consistency point, also referenced as a checkpoint. A consistency point is a point in time in the log at which a known and consistent state for the database system can be established. Typically, a consistency point involves recording to non-volatile storage (e.g., a disk) a certain amount of information such that if a failure occurs, the database system can restart at that established consistency point or can achieve a consistent state based on that established consistency point. This information comprises data associated with on-going processing and a record of the consistency point event. In a database, the data associated with on-going processing is transaction data; in other applications the data associated with on-going processing is known as message data, registers, etc.
A purpose of a consistency point is to periodically move forward in the log a logical recovery restart point for the database system. In the absence of consistency points, the database system is required to process all the transactions in the log recorded since the system was restarted, if a failure were to occur.
Conventional recovery schemes for database systems comprise a full-blocking consistency point and a non-blocking or “fuzzy” consistency point. Although these conventional recovery schemes have proven to be useful, it would be desirable to present additional improvements.
The full-blocking consistency point comprises a “hard” consistency point that writes out all dirty pages from the buffer pool. The full-blocking consistency point refers to the logging of additional information, and is often referred to as the undo log or physical log, which includes a physical logging of original images of all pages modified between consistency points such that, after a crash, the system can be restored to a physically consistent point that matches the state of the system at the completion of the last consistency point prior to a failure. With the system restored to the physically consistent point of the last consistency point, transaction recovery only has to apply those log records occurring after the last consistency point. From a recovery point of view, the full-blocking consistency point is efficient compared to conventional consistency points because all the data associated with the on-going processing is established at the same point in time, allowing an efficient restart of a system after failure.
While a full-blocking consistency point is simple, it suffers from transaction disruption and administration complexity. Transaction disruption occurs because users are blocked from performing update transactions while the buffer pool is being flushed. It is not uncommon to see systems that have been blocked for several minutes. Administration complexity occurs because the disruption, during which updates are blocked, can be quite lengthy. A common source of administrative work is attempting to configure the system to reduce the number to dirty pages at consistency point time, or to move data around to balance I/O of the restoration so the restoration can be performed faster, reducing the duration of the transaction disruption.
A fuzzy consistency point stops processing briefly to record a consistency point event. To enable the system to write the consistency point event, application data is flushed to stable storage periodically. Therefore, the fuzzy consistency point requires the ability to establish a consistent point for all the applications running. Typically, there is some partial-blocking mechanism that is able to record to stable storage each application state individually. Consequently, there may be more than one restart point, making the system restart point “fuzzy”. From the application point of view, the fuzzy consistency point is efficient compared to conventional consistency points because the blocking period is very short. The fuzzy consistency point is inefficient for establishing a consistent point because multiple application consistent points are required.
A recovery scheme comprising a fuzzy consistency point does not write all dirty pages from the buffer pool. Because not all dirty pages are written, transaction recovery starts at a point associated with the oldest dirty page and conditionally applies log records based on whether a page on disk is newer than the log record. While a fuzzy consistency point is fast, the fuzzy consistency point suffers from complexity, a slow recovery time, and complex I/O patterns that lead to a lack of predictability in recovery time.
The conventional approach comprising the fuzzy consistency point is complex. A large amount of code is associated with conditionally skipping the replay of the various log record types. Further complexity occurs if there are side effects such that the replay of a log record affects more than just the page directly referenced. If the page is newer than the log record, the reapplication of the activity to the page can be skipped; however, the side effects may need to be redone. Furthermore, while an approach comprising the fuzzy consistency point is not required to flush pages, code is included for examining the entire buffer pool just to find the oldest page from which logical recovery starts, adding to system complexity.
Recovery time is slow for a conventional approach comprising a fuzzy consistency point. Because logical recovery starts at an arbitrary point associated with the oldest unwritten page, the system often processes much more of the logical log than an approach that simply starts with log records after the last consistency point. Further, because the periodic fuzzy consistency points do not reduce the number of dirty pages to zero, additional pages require redo. Moreover, fuzzy consistency points do not flush the buffer pool. Both fuzzy and full blocking checkpoints aim at reducing the number of dirty buffers, for different reasons. Full blocking checkpoints aim at less dirty buffers to reduce the blocking during the checkpoint. Fuzzy checkpoints aim at reducing the dirty buffers to shorten recovery time by moving the restart point forward in the log. The I/O flushing patterns for blocking consistency points and non-blocking (fuzzy) consistency points are significantly different.
While recovery in a system using fuzzy consistency points is in general slower, recovery can often be “much” slower. Additional recovery time is required based on the amount of extra processing required. Extra processing is required if a particularly old page was present when a crash occurred. Many of the pages associated with the older transactions that may have already been flushed to disk but fuzzy checkpoint recovery processing, must at least check the page to see if the transaction has already been applied leading to unnecessary I/Os. While customers do not like to wait for recovery, they especially do not like being unable to predict when the system recovery is complete or being surprised by restart times that are occasionally much longer than normal.
A method of consistency point processing is desired that allows transactions to occur while the consistency point is being processed, yet provides the recoverability performance of a full-blocking consistency point. What is therefore needed is a system, a computer program product, and an associated method for implementing a partial-blocking consistency point in a database. The need for such a solution has heretofore remained unsatisfied.